So, I think I found the source of my issue: my DNS setting.

I had shut off and uninstalled SquidGuard, Squid and Snort (in that order) and still had no luck. As soon as I changed my DNS settings to Google (for example), I no longer have any latency issues. Put back Squid and Snort and still no latency.

So, it looks like something with OpenDNS is causing my problem.

Off to try another DNS to see if it's still running ok.